Computer Vision and Human Skin Colour

Computer vision based face and gesture recognition will allow future human computer interfaces to be more intuitive and user-friendly than traditional interfaces. A crucial processing step for the success of such systems is robust detection and tracking of faces and hands, which is frequently done by combining complementary cues, e.g., motion, shape, and colour. Skin colour is often used because it is invariant to orientation and size, gives an extra dimension compared to gray scale methods, and is fast to process. The main problems with the robustness of skin colour detection are however: (1) dependence on the illumination colour, (2) it varies between individuals, and (3) many everyday-life objects are skin colour like, i.e., skin colour is not unique. The objective of this study is to open for an improved skin colour cue, and the focus is to investigate the image formation process theoretically and experimentally – in particular with respect to human skin colours under changing and mixed illumination. Physics-based approaches are used to model the reflections of skin and the image formation process when registered by a camera. It is shown that skin colour “perception” as viewed by a state-of-the-art colour video camera can be modelled sufficiently accurate with a physics-based approach given the illumination spectra, the reflectance of skin, and the camera characteristics. Furthermore, everyday-life illumination spectra can be modelled appropriately as Blackbody radiators in this context. This skin colour modelling may provide the basis for applications such as adaptive skin segmentation. For adaptive segmentation it may also be useful to estimate the illumination colour. Two methods are suggested and tested to estimate the illumination colour from observation of skin colour. The first uses the diffuse reflections from skin and the second uses the surface or highlight reflections. These methods are complementary and their accuracies are sufficient to improve adaptive skin segmentation. In order to track skin areas through changing illumination conditions and to distinguish them from other skin coloured objects a method is proposed to model the skin colour distribution as a unimodal Gaussian. The parameters of the Gaussian can be modelled selectively for arbitrary illumination using a physics-based approach. Finally, the reflectance characteristics of skin in the near infrared (NIR) spectrum are explored. A combination of standard RGB bands with three narrow NIR bands is suggested to robustly detect skin under changing illumination and distinguish it from other skin colour-like objects. The results of this work may contribute to an adaptive skin colour cue that in combination with other cues will enable robust face and hand detection in unconstrained environments. The features of the skin colour cue, which combines the methods developed, are outlined in the last chapter of this report.